Intelligent control of nonlinear systems with actuator saturation using neural networks

Common actuator nonlinearities such as saturation, deadzone, backlash, and hysteresis are unavoidable in practical industrial control systems, such as computer numerical control (CNC) machines, xy-positioning tables, robot manipulators, overhead crane mechanisms, and more. When the actuator nonlinearities exist in control systems, they may exhibit relatively large steady-state tracking error or even oscillations, cause the closed-loop system instability, and degrade the overall system performance. Proportional-derivative (PD) controller has observed limit cycles if the actuator nonlinearity is not compensated well. The problems are particularly exacerbated when the required accuracy is high, as in micropositioning devices. Due to the non-analytic nature of the actuator nonlinear dynamics and the fact that the exact actuator nonlinear functions, namely operation uncertainty, are unknown, the saturation compensation research is a challenging and important topic with both theoretical and practical significance. Adaptive control can accommodate the system modeling, parametric, and environmental structural uncertainties. With the universal approximating property and learning capability of neural network (NN), it is appealing to develop adaptive NN-based saturation compensation scheme without explicit knowledge of actuator saturation nonlinearity. In this dissertation, intelligent anti-windup saturation compensation schemes in several scenarios of nonlinear systems are investigated. The nonlinear systems studied within this dissertation include the general nonlinear system in Brunovsky canonical form, a second order multi-input multi-output (MIMO) nonlinear system such as a robot manipulator, and an underactuated system-flexible robot system. The abovementioned methods assume the full states information is measurable and completely known. During the NN-based control law development, the imposed actuator saturation is assumed to be unknown and treated as the system input disturbance. The schemes that lead to stability, command following and disturbance rejection is rigorously proved, and verified using the nonlinear system models. On-line NN weights tuning law, the overall closed-loop performance, and the boundedness of the NN weights are rigorously derived and guaranteed based on Lyapunov approach. The NN saturation compensator is inserted into a feedforward path. The simulation conducted indicates that the proposed schemes can effectively compensate for the saturation nonlinearity in the presence of system uncertainty

A new kernel method for hyperspectral image feature extraction

Hyperspectral image provides abundant spectral information for remote discrimination of subtle differences in ground covers. However, the increasing spectral dimensions, as well as the information redundancy, make the analysis and interpretation of hyperspectral images a challenge. Feature extraction is a very important step for hyperspectral image processing. Feature extraction methods aim at reducing the dimension of data, while preserving as much information as possible. Particularly, nonlinear feature extraction methods (e.g. kernel minimum noise fraction (KMNF) transformation) have been reported to benefit many applications of hyperspectral remote sensing, due to their good preservation of high-order structures of the original data. However, conventional KMNF or its extensions have some limitations on noise fraction estimation during the feature extraction, and this leads to poor performances for post-applications. This paper proposes a novel nonlinear feature extraction method for hyperspectral images. Instead of estimating noise fraction by the nearest neighborhood information (within a sliding window), the proposed method explores the use of image segmentation. The approach benefits both noise fraction estimation and information preservation, and enables a significant improvement for classification. Experimental results on two real hyperspectral images demonstrate the efficiency of the proposed method. Compared to conventional KMNF, the improvements of the method on two hyperspectral image classification are 8 and 11%. This nonlinear feature extraction method can be also applied to other disciplines where high-dimensional data analysis is required

Delayed Stochastic Algorithms for Distributed Weakly Convex Optimization

This paper studies delayed stochastic algorithms for weakly convex optimization in a distributed network with workers connected to a master node. More specifically, we consider a structured stochastic weakly convex objective function which is the composition of a convex function and a smooth nonconvex function. Recently, Xu et al. 2022 showed that an inertial stochastic subgradient method converges at a rate of $\mathcal{O}(\tau/\sqrt{K})$, which suffers a significant penalty from the maximum information delay $\tau$. To alleviate this issue, we propose a new delayed stochastic prox-linear ($\texttt{DSPL}$) method in which the master performs the proximal update of the parameters and the workers only need to linearly approximate the inner smooth function. Somewhat surprisingly, we show that the delays only affect the high order term in the complexity rate and hence, are negligible after a certain number of $\texttt{DSPL}$ iterations. Moreover, to further improve the empirical performance, we propose a delayed extrapolated prox-linear ($\texttt{DSEPL}$) method which employs Polyak-type momentum to speed up the algorithm convergence. Building on the tools for analyzing $\texttt{DSPL}$, we also develop improved analysis of delayed stochastic subgradient method ($\texttt{DSGD}$). In particular, for general weakly convex problems, we show that convergence of $\texttt{DSGD}$ only depends on the expected delay

Optimized kernel minimum noise fraction transformation for hyperspectral image classification

This paper presents an optimized kernel minimum noise fraction transformation (OKMNF) for feature extraction of hyperspectral imagery. The proposed approach is based on the kernel minimum noise fraction (KMNF) transformation, which is a nonlinear dimensionality reduction method. KMNF can map the original data into a higher dimensional feature space and provide a small number of quality features for classification and some other post processing. Noise estimation is an important component in KMNF. It is often estimated based on a strong relationship between adjacent pixels. However, hyperspectral images have limited spatial resolution and usually have a large number of mixed pixels, which make the spatial information less reliable for noise estimation. It is the main reason that KMNF generally shows unstable performance in feature extraction for classification. To overcome this problem, this paper exploits the use of a more accurate noise estimation method to improve KMNF. We propose two new noise estimation methods accurately. Moreover, we also propose a framework to improve noise estimation, where both spectral and spatial de-correlation are exploited. Experimental results, conducted using a variety of hyperspectral images, indicate that the proposed OKMNF is superior to some other related dimensionality reduction methods in most cases. Compared to the conventional KMNF, the proposed OKMNF benefits significant improvements in overall classification accuracy

OptiMUS: Optimization Modeling Using MIP Solvers and large language models

Optimization problems are pervasive across various sectors, from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the-art solvers, as the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. We introduce OptiMUS, a Large Language Model (LLM)-based agent designed to formulate and solve MILP problems from their natural language descriptions. OptiMUS is capable of developing mathematical models, writing and debugging solver code, developing tests, and checking the validity of generated solutions. To benchmark our agent, we present NLP4LP, a novel dataset of linear programming (LP) and mixed integer linear programming (MILP) problems. Our experiments demonstrate that OptiMUS solves nearly twice as many problems as a basic LLM prompting strategy. OptiMUS code and NLP4LP dataset are available at \href{https://github.com/teshnizi/OptiMUS}{https://github.com/teshnizi/OptiMUS

Neuro Control of Nonlinear Discrete Time Systems with Deadzone and Input Constraints

A neural network (NN) controller in discrete time is designed to deliver a desired tracking performance for a class of uncertain nonlinear systems with unknown deadzones and magnitude constraints on the input. The NN controller consists of two NNs: the first NN for compensating the unknown deadzones; and the second NN for compensating the uncertain nonlinear system dynamics. The magnitude constraints on the input are modeled as saturation nonlinearities and they are dealt with in the Lyapunov-based controller design. The uniformly ultimate boundedness (UUB) of the closed-loop tracking errors and the neural network weights estimation errors is demonstrated via Lyapunov stability analysis

Boosting Method in Approximately Solving Linear Programming with Fast Online Algorithm

In this paper, we develop a new algorithm combining the idea of ``boosting'' with the first-order algorithm to approximately solve a class of (Integer) Linear programs(LPs) arisen in general resource allocation problems. Not only can this algorithm solve LPs directly, but also can be applied to accelerate the Column Generation method. As a direct solver, our algorithm achieves a provable $O(\sqrt{n/K})$ optimality gap, where $n$ is the number of variables and $K$ is the number of data duplication bearing the same intuition as the boosting algorithm. We use numerical experiments to demonstrate the effectiveness of our algorithm and several variants

Optimal Diagonal Preconditioning: Theory and Practice

Preconditioning has been a staple technique in optimization and machine learning. It often reduces the condition number of the matrix it is applied to, thereby speeding up convergence of optimization algorithms. Although there are many popular preconditioning techniques in practice, most lack theoretical guarantees for reductions in condition number. In this paper, we study the problem of optimal diagonal preconditioning to achieve maximal reduction in the condition number of any full-rank matrix by scaling its rows or columns separately or simultaneously. We first reformulate the problem as a quasi-convex problem and provide a baseline bisection algorithm that is easy to implement in practice, where each iteration consists of an SDP feasibility problem. Then we propose a polynomial time potential reduction algorithm with $O(\log(\frac{1}{\epsilon}))$ iteration complexity, where each iteration consists of a Newton update based on the Nesterov-Todd direction. Our algorithm is based on a formulation of the problem which is a generalized version of the Von Neumann optimal growth problem. Next, we specialize to one-sided optimal diagonal preconditioning problems, and demonstrate that they can be formulated as standard dual SDP problems, to which we apply efficient customized solvers and study the empirical performance of our optimal diagonal preconditioners. Our extensive experiments on large matrices demonstrate the practical appeal of optimal diagonal preconditioners at reducing condition numbers compared to heuristics-based preconditioners.Comment: this work originally appeared as arXiv:2003.07545v2, which was submitted as a replacement by acciden

CD44ICD promotes breast cancer stemness via PFKFB4-mediated glucose metabolism.

CD44 is a single-pass cell surface glycoprotein that is distinguished as the first molecule used to identify cancer stem cells in solid tumors based on its expression. In this regard, the CD44high cell population demonstrates not only the ability to regenerate a heterogeneous tumor, but also the ability to self-regenerate when transplanted into immune-deficient mice. However, the exact role of CD44 in cancer stem cells remains unclear in part because CD44 exists in various isoforms due to alternative splicing. Methods: Gain- and loss-of-function methods in different models were used to investigate the effects of CD44 on breast cancer stemness. Cancer stemness was analyzed by detecting SOX2, OCT4 and NANOG expression, ALDH activity, side population (SP) and sphere formation. Glucose consumption, lactate secretion and reactive oxygen species (ROS) levels were detected to assess glycolysis. Western blot, immunohistochemical staining, ELISA and TCGA dataset analysis were performed to determine the association of CD44ICD and PFKFB4 with clinical cases. A PFKFB4 inhibitor, 5MPN, was used in a xenograft model to inhibit breast cancer development. Results: In this report, we found that the shortest CD44 isoform (CD44s) inhibits breast cancer stemness, whereas the cleaved product of CD44 (CD44ICD) promotes breast cancer stemness. Furthermore, CD44ICD interacts with CREB and binds to the promoter region of PFKFB4, thereby regulating PFKFB4 transcription and expression. The resultant PFKFB4 expression facilitates the glycolysis pathway (vis-à-vis oxidative phosphorylation) and promotes stemness of breast cancer. In addition, we found that CD44ICD and PFKFB4 expressions are generally up-regulated in the tumor portion of breast cancer patient samples. Most importantly, we found that 5MPN (a selective inhibitor of PFKFB4) suppresses CD44ICD-induced tumor development. Conclusion: CD44ICD promotes breast cancer stemness via PFKFB4-mediated glycolysis, and therapies that target PFKFB4 (e.g., 5MPN therapy) may lead to improved outcomes for cancer patients